Description
This displays the resulting daily filled images calculated using the fill_gaps.R script.
Different parameters were tested on the following data (note there are 2 different days, one with good daily coverage and one without):
Region: Northwest Atlantic (NWA, 39 to 82 N, 42 to 95 W)
Sensor: MODIS
Resolution: 4km
Processing level: Level 3, binned (L3b)
Year: 2015
Days: 57, 172
Pixels outside 0-64 mg m^-3 removed
Days with < 5% coverage removed Filling the gaps in log space (and transforming the results back to linear space), we’ll compare the results of OCx and POLY4, and try adding an extra year of data on either side of the target year to check if a longer time series helps improve the reconstruction.
Each performance summary has the following:
- original image(s) vs filled image(s)
- 2 separate maps: one early in the year and one later, to check performance on days with low and medium/high spatial coverage
- linear regression of filled pixels vs original pixels (in log space) using randomly-spaced CV (cross-validation) pixels
- various metrics:
- RMSE and number of EOFs used
- R^2, p-value, number of observations (nobs), intercept, and slope of the cross-validation regression
- RMSE and number of EOFs used
- linear regression of in situ samples VS filled satellite data (in situ samples restricted to matches <= 10 km away)
Extra details
ImputeEOF removes randomly sampled valid pixels for cross-validation. The number of pixels used is the maximum of 30, or 10% of the pixels. The function continues adding EOFs and calculating the resulting RMSE between real and reconstructed cross-validation pixels until the difference between the current RMSE and RMSE of the previous iteration is below a certain threshold (i.e. adding the most recent EOF did not significantly improve the RMSE). The threshold, called the “tolerance”, is set to 0.001 here.
The linear regression uses the standard major axis method (SMA) from lmodel2::lmodel2(), since it minimizes the area of the triangle instead of the distance in the x or y direction alone (i.e. it assumes there is error in both the independent and dependent variables, the “real” and filled/reconstructed data).
The chla POLY4 algorithm uses the same formulation as OCx, but with coefficients tuned to the NWA to remove the bias (whereas OCx is tuned with global in situ data). Therefore, it provides a better fit with the in situ data in the NWA.
Hilborn and Costa (2018) found that pixel reconstruction improved with more data in a smaller region on the Canadian Pacific coast, so we will test this in the larger NWA region. Note that the 3year DINEOF runs use the same cross-validation pixels for a single year (2015 alone) with extra randomly-selected pixels from the remaining years. Also, the CV regression is performed using only the CV pixels for 2015 to give a more accurate comparison between methods.
An analysis of DINEOF on the Canadian Pacific coast:
Hilborn A, Costa M. Applications of DINEOF to Satellite-Derived Chlorophyll-a from a Productive Coastal Region. Remote Sensing. 2018; 10(9):1449. https://doi.org/10.3390/rs10091449
OCx
1 year
Number of EOF: 13
Total RMSE: 0.2024255
Day 57 RMSE: 0.2176204
Day 172 RMSE: 0.171977
3 years
Number of EOF: 17
Total RMSE: 0.1904115
Day 57 RMSE: 0.2156396
Day 172 RMSE: 0.1889701
Summary
Number of EOFs for 1/3 years: 13/17
POLY4
1 year
Number of EOF: 16
Total RMSE: 0.2302663
Day 57 RMSE: 0.302537
Day 172 RMSE: 0.2139626
3 years
Number of EOF: 23
Total RMSE: 0.2188391
Day 57 RMSE: 0.2966093
Day 172 RMSE: 0.2333929
Summary
Number of EOFs for 1/3 years: 16/23